Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Core motion implementation for iPhone (Accelerometer/Gyro/Magnetometer support) #7127

Merged
merged 5 commits into from
Jan 14, 2017

Conversation

BastiaanOlij
Copy link
Contributor

These changes replace the now deprecated UIAccelerometer implementation with one using the newer CoreMotion, specifically implementing CMDeviceMotion. This adds gyro and magnetometer data to the iPhone platform.

I've gone for a simple polling approach, CMDeviceMotion does have an option for pushing data but I'm not sure if there is any added benefit.

The accelerometer data is split into a user vector and gravity vector by iOS but I add them together again so it behaves as before.

The gyro seems to work but testing an app that was build for the Android platform the axis seem flipped. Without an Android device to compare output with I'm not entirely sure how to best deal with this.

I have not been able to test the magnetometer output, my iPhone 5 doesn't seem to be giving any output though that may be a fault on my side in testing.

@BastiaanOlij
Copy link
Contributor Author

I'm still trying to figure out the differences between the Android and iPhone platform. It's a bit of a puzzle especially since I'm pretty new to this and have no Android experience but obviously only have access to Android based examples in Godot.

One thing that stands out to me is that the documentation for Android speaks of returning the gyroscope data in radians per second (rate of rotation) while the iPhone document is rather indiscriminate about it but if I read it correctly its reflected as the radians of rotation since last the last time it was requested.

Also now that I've read through the sourcecode of the android implementation I noticed the axis are adjusted depending on the rotation of the display which makes sense. It is an adjustment I'm not currently making on iPhone.

One option that I find very interesting on iPhone is the attitude property of the CMDeviceMotion class. This property gives access to absolute rotation data. Basically it's core motion doing all the work for us in processing the 9-axis data and ending up with absolute values. That would be very interesting to expose but in the spirit of being cross platform, I don't have a clue how to make the same available on Android.

@BastiaanOlij
Copy link
Contributor Author

ok, productive little night. I've added some code to handle flipping the x and y variables depending on the orientation of the screen similar to what was implemented on Android.

I have also removed making the Z negative, after outputting a bunch of things to the screen I came to the conclusion Z aligns itself with Godots coordinate system.

Finally I now know that the rotation rates on iPhone are indeed also in Radians per second just like on Android.

I have not added support for using attitude yet, I'm thinking about tackling that some time during the week.

@punto-
Copy link
Contributor

punto- commented Nov 19, 2016

This is awesome, thanks for all the work! as you mentioned, the first
priority is to make it portable, ie the same game code will behave the same
in both Android and iOS. Eventually we can add something to get raw values
depending on the platform if someone needs it, but it's not a priority
right now.

@BastiaanOlij
Copy link
Contributor Author

Punto,

It's not the raw data I'm after, its the processed data that gives you a rotation matrix for the device orientation. Now we need to take the gyro/accelerometer/magnetometer data and construct that ourselves in GDScript.

I'll probably add a get_deviceOrientationMatrix method to Input which returns this matrix when available and else calculates it based on the available data (if I find some good code to do so).

I'm also thinking about adding a get_gravity as both Android and iPhone support extracting the gravity vector from the accelerometer and its handy to have as a separate vector.

@akien-mga akien-mga added this to the 3.0 milestone Nov 21, 2016
@BastiaanOlij
Copy link
Contributor Author

I have added support for retrieving the gravity vector from iOS. This can be obtained by calling Input.get_gravity. When subtracted from the accelerometer data you can get the pure user movement of the device.

I have not implemented this on Android as I do not have access to an Android device. Someone will need to add that. This should be pretty straight forward as Android supports this. Simply duplicate whatever is being done for TYPE_ACCELEROMETER with TYPE_GRAVITY. I do not want to make these changes blind. Read more here:
https://developer.android.com/guide/topics/sensors/sensors_motion.html

I've also put a test godot project that implements all these changes here:
https://dl.dropboxusercontent.com/u/98707/Godot/TestCoreMotion.zip
This test project basically shows the output of the gyro, accelerometer, gravity and magnetometer on screen and draws a 3D box that is rotated as you move your phone around.

@BastiaanOlij
Copy link
Contributor Author

I have fixed a bunch of issues in my sample project. It is now available on github as well:
https://github.com/BastiaanOlij/TestCoreMotion
It also now updates the camera orientation so the effect is like looking around by moving your phone. The code is directly usable for AR/VR simply by adding a stereo camera (see my other project).

This project can be used to test the changes I've made here. Note that for some reason, either a bug on my part or my aging iPhone, the magnetometer doesn't give any output so the code related to that is completely experimental.

I would appreciate it if someone would test this on Android especially if you have access to both platforms to see if the project behaves the same. On android currently the accelerometer is ignored and does not compensate drift. This is because as I mentioned up above, on Android someone will need to implement TYPE_GRAVITY to extract the gravity vector from the accelerometer data. Using the accelerometer data will cause jitter as any shaking of the device will translate to jitter in the UI.

@akien-mga
Copy link
Member

@BastiaanOlij You will have to stretch your git muscles again to fix the merge conflict here too :)

@punto- Could you give it a try then and merge if it's good?

@BastiaanOlij
Copy link
Contributor Author

@akien-mga , @punto- , while it will be easy to fix up the merge conflicts the big issue here is that we can't test any of the changes until we complete PR #7457. We need a working iOS build first and @reduz mentioned that me and @punto- should have a closer chat to see what needs to be done to get the new renderer to work on ios :)

@BastiaanOlij
Copy link
Contributor Author

I think I've got it rebased properly


if (motionInitialised) {
///@TODO is this the right place to clean this up?
[motionManager stopDeviceMotionUpdates];
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indentation ;)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It'll be fixed when I apply clang-format anyway, so let's merge :p

@@ -253,11 +271,24 @@ - (void)applicationDidFinishLaunching:(UIApplication*)application {
[window makeKeyAndVisible];

//Configure and start accelerometer
/*
Old accelerometer approach deprecated since IOS 7.0
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd say go ahead and remove the obsolete code, I don't think Apple will be reintroducing obsoleted APIs in future iOS versions :)

@akien-mga
Copy link
Member

I think we could merge as is, with the current master branch it's hard to test but the code looks good.

@BastiaanOlij Could you open an issue about ensuring that this API works similarly for Android and iOS before 3.0 is released? + maybe if you're up to making an official demo of using those features so that it can be tested on both platforms :) This would likely have to wait until master is usable to produce such a demo, hence the need for an issue as reminder ;)

akien-mga added a commit to akien-mga/godot that referenced this pull request Nov 27, 2020
It has been implemented for iOS a long time ago already with godotengine#7127.
akien-mga added a commit to akien-mga/godot that referenced this pull request Dec 2, 2020
It has been implemented for iOS a long time ago already with godotengine#7127.

(cherry picked from commit 2d0fda3)
HEAVYPOLY pushed a commit to HEAVYPOLY/godot that referenced this pull request Dec 14, 2020
It has been implemented for iOS a long time ago already with godotengine#7127.

(cherry picked from commit 2d0fda3)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants